METHOD OF PROCESSING IMAGE SIGNAL FROM SOLID-STATE 
IMAGING DEVICE, IMAGE SIGNAL PROCESSING APPARATUS, 
IMAGE SIGNAL GENERATING APPARATUS AND COMPUTER PROGRAM 
PRODUCT FOR IMAGE SIGNAL PROCESSING METHOD 

BACKGROUND OF THE INVENTION 

The present invention relates to a method of 
processing the color component signal (image signal) 
obtained from a solid-state image pickup device, an 
image signal processing apparatus, an image signal 
generating apparatus comprising the image signal 
processing apparatus and the solid-state image pickup 
device, and a computer program product for the image 
signal processing method. 

The solid-state image pickup device or the 
image sensor has a two-dimensional arrangement of a 
plurality of photoelectric elements such as 
photodiodes . For example, both the three primary color 
signals such as R, B and G signals and the luminance 
signal are produced in a single-chip solid-state image 
pickup device. A monochromatic color filter of R (red 
light penetrating) , G (green light penetrating) or B 
(blue light penetrating) is arranged on the light- 
receiving surface of each photoelectric element. A 
white filter (a filter of white light penetrating) is 
arranged in place of the G filter in some cases. A 
plurality of photoelectric elements including these 
three types of color filters are arranged on the light- 



receiving surface in a predetermined pattern such as 
Bayer arrangement pattern. Hereinafter, we will 
explain examples wherein each photoelectric element 
corresponds to a pixel. The solid-state image pickup 
5 device is of either CCD type or MOS FET type. 

Generally, the color component signals 
composed of all the three primary colors are required 
from each pixel for producing high quality full-color 
image signals. In the single-chip solid-state image 
^ 10 pickup device, however, only a monochromatic color 

component signal corresponding to the color of the 
.2 filter of a respective photoelectric element of a pixel 

r7 can be produced in each pixel. In each pixel of a 

given color in the single-chip solid-state image pickup 
O 15 device the two remaining color component signals are 

iU produced in such a manner that the signals from the 

O adjacent pixels having different color filters are 

O 

subjected to the signal processing such as 
interpolation . 

20 An example of the image signal processing 

method as described above is disclosed in US Patent No. 
5,382,976. This US patent describes a signal 
processing method for a single image sensor including a 
red pixel, a green pixel and a blue pixel. For the red 

25 pixel or the blue pixel not including the green 

component, the green value is obtained by interpolating 
the signal values of the pixels adjacent to the 
particular pixel. Also, the US patent includes the 
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description to the effect that with regard to the green 
value of phase "00" containing the green value or phase 
"11", the adjacent pixel values are not interpolated 
but the signal values of the phases are used as they 
5 are without interpolation as indicated by G = G(0,0) in 
Figs. 4A and 4B of the U.S. patent. 

Now, a specific example of an image signal 
processing method will be explained with reference to 
the drawings . 

10 Fig. 2 is a diagram showing an example a 

block configuration of an image signal processing 
apparatus comprising a single-chip solid-state image 
pickup device and various processing circuits for 
processing the image signal picked up by the solid- 

15 state image pickup device. In Fig. 2, reference 

numeral 1 designates a solid-state image pickup device 
such as a charge-coupled device (CCD) image sensor for 
converting the imaging light into electric charge and 
outputting it as a video signal, numeral 2 a sample- 

20 hold and automatic gain control (CDS&AGC) circuit {CDS: 
Correlated Double Sampling) for sample holding the 
video signal outputted from the CCD 1 and outputting 
the video signal by amplifying the sample hold signal 
to a required level, and numeral 3 an A/D converter for 

25 converting the analog video signal from the CDS&AGC 
circuit 2 into a digital video signal. 

Numeral 4 designates a DSP (digital signal 
processor) circuit for processing the video signal from 
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the A/D converter 3 as required. 

The color filters of the solid-state image 
pickup device (CCD) 1 are arranged in Bayer arrangement 
pattern. Fig. 3A shows the Bayer arrangement pattern 
5 and the arrangement of the pixel addresses attached to 
the pixels. In Fig. 3A, a green light transmitting 
filter (a filter of green light penetrating) is 
designated as G, a red light transmitting filter (a 
filter of red light penetrating) as R and a blue light 

10 transmitting filter (a filter of blue light 

penetrating) as B. Further, the number 1 is attached 
to the highest row and the leftmost column. For 
example, the pixel address of a pixel having a green 
light transmitting filter located on the mth row from 

15 top and the nth column from left is expressed as G^.^^- 
The symbol of the pixel address is also expressed as 
the value of the color component signal obtained by 
picking up an image in the particular pixel. 

A method of image signal processing for the 

20 green component signal constituting one of the 

processes executed in the DSP circuit 4 described above 
will be explained with reference to Fig. 4C showing an 
example of the pixel interpolation method. The method 
(i) or (ii) described below is adaptively used for each 

25 pixel address. Specifically, in method (i), as for the 
green component signal level of a pixel address (e.g. 
G22) having a green light transmitting filter, the value 
of the green component signal level of the particular 



pixel address is used as it is. In method (ii) , on the 
other hand, as for the green component signal level of 
a pixel address (e.g. R2.) having a red light 
transmitting filter or the green component signal level 
5 of a pixel address (e.g. B.J having a blue light 

transmitting filter, the average value of the green 
component signal levels obtained from the pixels (e.g. 
Gi3/ G22/ G24, G33) having a green light transmitting 
filter at the upper, lower, left and right adjacent 
10 pixel addresses is used. 

Q 

ifl A specific example of the signal processing 

Ul 

■^-j method (ii) described above will be explained in more 

detail. As shown in Fig. 4C, in the case where the 
image signal is processed for the pixel address R23 to 

^ 15 generate the green component signal level g23 by 

interpolation, for example, the value is calculated 
from 

° 923 = (Gi3 + G22 + G24 + G33)/4 (1) ■ 

Another method of calculation included in the 
20 signal processing method (ii) uses the values of only 

the left and right adjacent pixels of the pixel address 
R23 as shown in equation (2) below for calculation. 

g23 = (G22 + G2J /2 (2) 
Various other alternative methods are conceivable 
25 including a method in which only the upper and lower 
adjacent pixel values or only the diagonally adjacent 
pixel values of the pixel address R23 are used for 
calculation . 
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Fig. 4A is a diagram showing an interpolation 
method for the R component signal at the position of 
the pixel address G22 . In this case, the red component 
signal at the position of 0,2 has a pixel level equal to 
5 the average of the imaging red component -signal levels 
of the pixel address R^i and the pixel address R,.. Fig. 
4B is a diagram showing an interpolation method for the 
B component signal at the position of the pixel address 
G22 • In this case, the blue component signal at the 
10 position of G22 has a pixel level equal to the average 

O 

'^u of the imaging blue component signal levels of the 

"-J pixel address B^, and the pixel address B32 - Further, 

Fig. 4D is a diagram showing an interpolation method 
m for the B component signal at the position of the pixel 

13 15 address R23. In this case, the blue component signal at 

the position of R23 has a pixel level equal to the 
]S average of the imaging blue component signal levels at 

^ the respective positions of the pixel addresses B12/ 

B32 and B34 . 

20 The signal processing method described above 

has the following problem. Specifically, assume that 
the band limiting range of the source follower circuit 
of the amplifier 15 of the output unit of the CCD used 
for securing the required S/N ratio, the bandwidth of 

25 the CDS circuit 2 or the band limiting range of the 
process amplifier is excessively narrow. Then, the 
waveform of the output signal is distorted in each 
case. As a result, a horizontal line having the blue 



light transmitting filter such as the (m-l)th line 
(row) in Fig. 3A and the horizontal line having the red 
light 'transmitting filter such as the mth line (row) in 
Fig. 3A have different waveform distortion amounts of 
5 the green component signal . Thus, the error levels 
applied to the green component signals are different 
between adjacent lines, often resulting in variations 
of the green component signal level, even when a 
uniform green component light in the imaging light is 

10 applied from the object to each pixel. The green 
component signals varied from one line to another 
causes horizontal stripes of noises on the image 
- subjected to the signal processing as shown in Fig. 4C. 

Also, in the case where the conditions for 

15 the production process undergo a change, the filter 

characteristic of the green light transmitting filter 
among the CCD color filters may be varied in every 
other horizontal scanning line, i.e. between a 
horizontal line having the blue light transmitting 

20 filter and a horizontal line having the red light 

transmitting filter, which often causes variations of 
the green component signal level between different 
lines . 

This often leads to the problem that even 
25 when a uniform green component light in the imaging 
light is applied from an object to each pixel, the 
levels of the green component signals corresponding to 
the pixels of the green light transmitting filters 
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outputted from the CCD may be varied between even-row 
horizontal lines and odd-row horizontal lines. 

Figs. 6A and 6B show waveforms obtained by 
sampling the output signal of the CCD in a correlated 
5 double sampling (CDS) circuit when the CCD having the 
aforementioned problem picks up an object having a 
uniform brightness and color in a whole picture frame. 
Fig. 6A shows a signal waveform of a horizontal line 
(the mth horizontal line in Fig.*3A, for example) 
10 having a red light transmitting filter and green one, 
and Fig. 6B shows a signal waveform of a horizontal 
line (the (m-l)th horizontal line in Fig. 3A, for 
example) having a blue light transmitting filter and 
green one. 

15 In Figs. 6A and 6B, numerals 100, 200 

designate reference voltage signal (for example, an 
optical black signal) waveforms. Numeral 110 
designates an image signal waveform corresponding to 
the reference voltage signal waveform 100, and numeral 

20 210 designates an image signal waveform corresponding 
to the reference voltage signal waveform 200. The 
values of the color component signals of the image 
signal waveforms 110, 210 are indicated by the level 
difference with the reference voltage signal waveforms 

25 100, 200, respectively. In the waveform 110, from left 
to right, the level waveforms of the red component 
signal, the green component signal and the red 
component signal are shown, while in the waveform 210, 
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from left to right, the level waveforms of the green 
component signal, the blue component signal and the 
green component signal are shown. 

Comparison between the green component signal 
5 of the waveform 110 in Fig. 6A and the green component 
signal of the waveform 210 in Fig. 6B shows that the 
signal levels are different between them due to the 
problem described above. In the case where an image is 
displayed on a monitor (not shown) inserted in a 

10 subsequent stage, this level difference is seen to 

cause a level difference between horizontal lines even 
in a displayed image produced by imaging an object 
uniformly. Thus the image is displayed in horizontal 
stripes or speckles, thereby extremely deteriorating 

15 the image quality. 



SUMMARY OF THE INVENTION 

The object of the present invention is to 
provide means for preventing the unnecessary level 
difference from being caused between horizontal lines 

20 or reducing the unnecessary level difference even in 
the case where the picking up characteristics of the 
CCD are varied or the performance thereof is 
deteriorated and thereby to prevent the deterioration 
of the image quality which otherwise might be caused by 

25 horizontal stripes or speckles on a picked-up image or 
a part of image by the imaging light from the object. 

According to one aspect of the invention. 



there is provided an image signal processing method for 
processing the color component signals obtained from a 
solid-state image pickup device including an 
arrangement of a plurality of photoelectric elements 
5 with color filters arranged in the light receiving 

section of the photoelectric elements, comprising the 
steps of fetching a color component signal from a 
designated pixel corresponding to the photoelectric 
element located on a line and having a filter that can 

10 transmit the green light in the solid-state image 
pickup device, fetching the color component signals 
from the pixel corresponding to the photoelectric 
element on other pixel line than the first line of the 
designated pixel in the neighborhood of the designated 

15 pixel, the neighboring pixels each having a filter for 
transmitting at least the green light, and determining 
the value of the color component signal of the 
designated pixel corresponding to the photoelectric 
element having the filter that can transmit at least 

20 the green light, based on a plurality of the color 
signals fetched in the foregoing steps. 

According to another aspect of the invention, 
there is provided an image signal processing apparatus 
for processing the color component signals obtained by 

25 a solid-state image pickup device including an 

arrangement of a plurality of photoelectric elements 
and a color filter arranged in the light receiving 
section of each pixel corresponding to the 
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photoelectric element comprising means for fetching the 
color component signal from a designated pixel 
corresponding to the photoelectric element having a 
filter that can transmit at least the green light on 
5 the lines of the solid-state image pickup device, means 
for fetching the color component signal from a pixel 
corresponding to the photoelectric element on other 
pixel line than said line of the designated pixel in 
the neighborhood of the designated pixel, the 

10 neighboring pixels each having a filter for 

transmitting at least the green light, and arithmetic 
means for determining the value of the color component 
signal of the designated pixel corresponding to the 
photoelectric element having the filter that can at 

15 least the green light, based on a plurality of the 
color signals fetched. 

According to still another aspect of the 
invention, there is provided an image signal generating 
apparatus comprising a solid-state image pickup device 

20 including an arrangement of a plurality of 

photoelectric elements and a color filter arranged in 
the light receiving section of each pixel corresponding 
to the photoelectric element, and an image signal 
processing unit for processing the color component 

25 signals obtained by the solid-state image pickup 

device, wherein the signal processing unit includes 
means for fetching the color component signal from a 
designated pixel corresponding to the photoelectric 
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element located on a line of the solid-state image 
pickup device and having a filter that can transmit at 
least the green light, means for fetching the color 
component signals from the pixel corresponding to the 
5 photoelectric element located on other lines in the 
neighborhood of the designated pixel, the neighboring 
pixels each having a filter capable of transmitting at 
least the green light, and arithmetic means for 
determining the value of the color component signal of 
10 the designated pixel corresponding to the photoelectric 
'^D element having a filter capable of transmitting at 

%l least the green light, based on a plurality of the 

1=:^ color signals fetched, 

^ According to a further aspect of the 

Q 15 invention, there is provided a computer program product 

^ comprising a computer usable medium having computer 

readable program code means embodied therein for 
processing the color component signals obtained by a 
solid-state image pickup device having an arrangement 
20 of a plurality of photoelectric elements with a color 
filter arranged in the light receiving section of each 
of the pixels corresponding to the photoelectric 
elements, wherein the computer readable program code 
means includes means for fetching the color component 
25 signal from a designated pixel corresponding to the 

photoelectric element located on a line of the solid- 
state image pickup device and having a filter that can 
pass at least the green light, means for fetching the 



color component signals from the pixel corresponding to 
the photoelectric element in the neighborhood of the 
designated pixel and located on line different from the 
designated pixel, the neighboring pixels being located 
5 on lines and each having a filter capable of 

transmitting at least the green light, and means for 
determining the value of the color component signal of 
the designated pixel corresponding to the photoelectric 
element having a filter capable of passing at least the 
10 green light, based on a plurality of the color signals 

O 

=5 fetched. 

BRIEF DESCRIPTION OF THE DRAWINGS 
m Figs. lA and IB are diagrams for explaining 

O an example of an image signal processing method 

fy 15 according to the invention. 

Fig. 2 is a diagram showing an example block 
configuration of an image signal processing apparatus. 

Figs. 3A and 3B are diagrams showing the 
Bayer arrangement pattern and another arrangement of 
20 filters, i.e. pixel addresses attached to the pixels. 

Figs. 4A, 4B, 4C and 4D are diagrams for 
explaining an example of the pixel interpolation 
method. 

Figs. 5A and 5B are diagrams showing the 
25 level of the green component signal of each pixel for 
the image signal processing method according to the 
invention . 



Figs. 6A and 6B are diagrams showing 
waveforms for horizontal lines containing the pixels R 
and B lines sampled from the CCD output signal in a 
correlated double sampling circuit. 

Fig. 7 is a block diagram showing an image 
signal generating apparatus including a signal 
processing apparatus according to an embodiment of the 
invention . 

Fig. 8 is a block diagram showing an image 
signal generating apparatus including a signal 
processing apparatus according to another embodiment of 
the invention . 

Fig. 9 shows an example of the main flowchart 
for interpolation of the green component signal in a 
signal processing apparatus according to the invention. 

Fig. 10 shows an example of the flowchart for 
the signal processing performed in a signal processing 
apparatus according to this invention as shown in Fig. 
lA or Fig. IB, 

Fig. 11 shows an example of the flowchart for 
the signal processing performed by a signal processing 
apparatus according to this invention even in the case 
of a filter arrangement other than the Bayer 
arrangement pattern shown in Fig. 3B. 

Fig. 12 is a flowchart schematically showing 
the signal processing steps for a signal processing 
apparatus according to the invention. 
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DESCRIPTION OF THE EMBODIMENTS 

Embodiments of the invention will be 
explained below. A block configuration of an image 
signal processing apparatus according to the invention 
5 is shown in Fig, 7. In Fig. 1, the same reference 
numerals as those in Fig. 2 designate the same 
component elements as the corresponding component 
elements, respectively, in Fig. 2. Also, the color 
filters of a CCD 1 are arranged in a manner similar to 

10 the Bayer arrangement pattern shown in Fig. 3A, and the 
indication of the pixel addresses and the symbol of a 
pixel address is expressed as the value of the color 
component signal obtained by picking up an image in the 
particular pixel. 

15 An image signal processing method, i.e. a 

method of processing the green component signal which 
is one of the processes carried out in a DSP circuit 40 
according to the invention will be explained with 
reference to Figs. lA, IB and the flowcharts of Figs. 9 

20 to 12 showing an example of the image signal processing 
method according to the invention. 

Fig. 9 shows the main routine of the image 
signal processing in the DSP 40 of a signal processing 
apparatus according to an embodiment of the invention. 

25 First, in step 101, the video signals of all the pixel 
addresses of an image frame to be interpolated are 
stored in a frame buffer memory 41 from an A/D 
converter 3. As an alternative, the video signal of a 
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pixel address G^^^, corresponding to a green light 
transmitting filter of the image frame is stored in the 
frame buffer memory 41, 

In step 102, an interpolation process is 
5 carried out in an interpolation processing unit 42 for 
determining the green component value among the three 
color component signal values at the respective 
positions of the green pixel, the red pixel and the 
blue pixel.. Hereinafter, the green pixel means a pixel 

10 corresponding to the photoelectric element provided 

with a color filter transmitting at least green light, 
the red pixel means a pixel corresponding to the 
photoelectric element provided with a color filter 
transmitting red light, and the blue pixel means a 

15 pixel corresponding to the photoelectric element 

provided with a color filter transmitting blue light. 
The green pixel may be replaced by a white pixel having 
a white filter which transmits green light and other 
particular color component light, A specific flow of 

20 the process performed in step fb2 will be explained 
with reference to the interpolation processing flows 
shown in Figs. 10 and 11. In step 103, it is 
determined whether the interpolation processing is 
complete for all the pixel addresses in an image frame 

25 intended for interpolation, and the process of steps 
102 and 103 is repeated until the value of the green 
component is determined for all the pixel addresses. 

Now, a specific subroutine of step 102 will 
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be explained. 

First, in the interpolation method shown in 
Fig. lA, the signal value of the green pixel is 
determined by method (i') described below in accordance 
5 with the pixel address. Further, method (ii) is 
adaptively applied for each of red and blue pixel 
addresses . 

(i') The green component signal of a pixel 
address G^^^ having a green light transmitting filter is 
10 the average of the value of the green component signal 

'-fl obtained for the particular pixel address G^^^ the 

111 

'-4 value of the green component signal picked up by the 

1=^ pixel of the pixel address Gj^-i^n-i the immediately 

m left column on the immediately upper row than the pixel 

O 15 address G^^^, 

. 

III In a specific example of application of 

J4 method (i') described above, as shown in Fig. lA, 

consider the case in which a green component signal 
level g22 is obtained by the processing the image signal 
20 for the pixel address G22. The value is calculated from 
g22 = (Gil + G22)/2 (3) 
This equation can be rewritten into a general 
expression below. 

g.n = (G.-i,n-i + G,,J/2 (4) 
25 In the method shown in Fig. IB, on the other 

hand, the method (i") described below is carried out in 
accordance with the pixel address followed by 
adaptively carrying out the method (ii) described 



above . 

(i") The green component signal of a pixel 
address G^,„ having a green light transmitting filter is 
a weighted average of the value of the green component 
signal picked up in the particular pixel and the values 
of the green component signals obtained in the pixels 
having a green light transmitting filter at the four 
pixel addresses G,.i,,.i, G^.,^,,,, G^.i,n-i^ G^.i.n.i diagonally 
upper and lower than the particular pixel address G^^^. 

As a specific example, as shown in Fig. IB, 
for example, assume that the green component signal 
level g22 is to be obtained by image signal processing 
for the pixel address G22- The value is calculated as 
shown below, 

g22 = (Gil + + G22 + G31 + G33)/5 (5) 

or by weighting pixel address G22 more than other pixel 
addresses as shown below. 

522 = (Gn + Gi3 + 4G22 + G31 + G33)/8 (6) 
Equations (5) and (6) can be rewritten into general 
expressions shown below. 

£G,,i,,,i)/co, (7) 
where a)=a+)3 + r+£. 

Fig, 10 is a flowchart for the aforementioned 
interpolation processing method according to an 
embodiment. This flow is a flow of the interpolation 
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processing of step 102 in the main flow shown in Fig. 
9. First, in step 201, the pixel address of the pixel 
intended for interpolation is designated. Assume that 
the designated pixel address is G^^,^^ (m, n: positive 
5 integers) . In step 202, it is determined whether the 
designated pixel address is that of the pixel having a 
green (white) light transmitting filter or not. In the 
case where the designated pixel is not a green (white) 
pixel, the pixel data is read from the frame memory 41 
10 and the data values of the red and blue pixels are 
determined by equation (1), (2) or (2') in step 203. 
In the case where the designated pixel is a green 
(white) pixel, on the other hand, the pixel data are 
read from the frame memory 41 and the data value of the 
15 green pixel is determined according to equation (4) or 
(7) in step 204. 

Various other calculation methods than the 
methods (i') and (i") are conceivable for the image 
processing method according to the invention. 
20 In the pattern of Fig. lA, for example, the 

green signal value of the designated pixel address G^^^ 
may be determined using the average value of the upper 
right pixel address G^.i,„,i and the lower left pixel 
address G^^^^^_^ each adjacent to the pixel address G^^„. 
25 Also, the green pixel on a horizontal line other than 
the upper and lower adjacent horizontal lines can be 
used as the green signal value of the designated pixel 
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The signal processing method and apparatus 
according to the invention described above are not 
limited to the image pickup device having the Bayer 
arrangement pattern, but is also applicable to other 
5 patterns of arrangement. For example. Fig. 11 shows a 
flowchart of interpolation for the color filter 
arrangement pattern shown in Fig. 3B. This flow is a 
routine in step 102 of the main flow shown in Fig. 9. 
First, the pixel address of the pixel 
10 intended for interpolation is designated in step 301. 
-:0 Assume that the designated pixel address is G^^„ (m, n: 

S| positive integers) . In step 302, it is determined 

M= whether the designated pixel address is that of a pixel 

m having a green (white) light transmitting filter or 

□ 15 not. In the case where the designated pixel is not a 

^ green (white) one, the pixel data is read out from the 

^ frame memory 41 and the data values of the red and blue 

pixels are determined according to equation (2) in step 
303. In the case the designated pixel is a green 
20 (white) pixel, on the other hand, the pixel data is 

read out from the frame memory 41 and the data value of 
the green pixel is determined in accordance with 
equation (4) or (8) in step 304. Here, equation (8) is 
disclosed below. 



25 



+ HG,,2,n)/W (8) 
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, where cx) = a+iS + r+ SrE. 

Fig. 12 is a flowchart showing the 
interpolation processing of step 102 in Fig. 9, i.e. 
the flow of Figs. 10 and 11 in broader term. Steps 
5 401, 402 are similar to steps 201, 202, respectively, 
in Fig. 10. In step 403, the green component signal 
value g^n the red pixel and the green component 
signal value g^,^ in the blue pixel are determined using 
the green (white) video signal component value obtained 
_ 10 from the pixel of a green (white) light transmitting 

=Q filter among those pixels adjacent to the pixel of the 

L§ a 

designated pixel address or the average of the 
predetermined green (white) component signal value. 
m Further, in step 404, the green component signal value 

p 15 g^^ of the green pixel is determined from the average of 

nj the green (white) component signal value obtained from 

the pixel of the designated pixel address and the green 
~ (white) component signal value obtained from at least 

one pixel of green (white) light transmitting filter 
20 among those pixels adjacent to the particular 
designated pixel. 

The behavior of the image signal level in the 
case of the image processing is conducted using an 
example of the image processing method according to the 
25 invention described above will be explained with 

reference to Figs. 5A and 5B. Figs. 5A and 5B show the 
level of the green component signal of each pixel while 
omitting the level of the red component signal and the 
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blue component signal. 

Figs. 5A and 5B are diagrams showing, in a 
matrix for each pixel address, the behavior of the 
green component signal {image signal) before and after 
5 the processing of an image picked up from a uniform 
object using the method of Fig. lA according to the 
invention. Fig. 5A shows the image signal level picked 
up before the image processing according to the 
invention, and Fig. 5B shows the image signal level 

10 obtained as the result of the image processing 
according to the invention. 

Fig. 5A shows the manner in which the green 
component signal level assumes different values of 140 
and 100 between odd rows, i.e. horizontal lines having 

15 a blue light transmitting filter and even rows, i.e. 
horizontal lines having a red light transmitting 
filter. In contrast. Fig. 5B shows the manner in which 
the image signal processing is conducted so that the 
green component signal level value is 120 for both odd 

20 rows, i.e. the horizontal lines having a blue light 

transmitting filter and even rows, i.e. the horizontal 
lines having a red light transmitting filter. 

The reason why the value of the green 
component signal level is 120 for both cases is that 

25 the image signal processing by the use of the method 
shown in Fig. lA produces a green component signal 
level value 120 obtained by an average of 140 and 100 
which are signal levels before the interpolation. 



Also, as for the values of the pixels having either a 
red light transmitting filter or a blue light 
transmitting filter, the green component signal level 
value assumes 120 as the result of using the 
aforementioned interpolation technique . 

The foregoing description of embodiments 
refers to the case in which the image signal processing 
according to the invention is performed in the DSP 
circuit 40. Nevertheless, the method according to the 
invention can be implemented in other than the DSP 
circuit 40. As shown in Fig. 8, for example, a 
personal computer 43 in which a picked up image signal 
is input can perform the image processing according to 
the invention and records the processing result or 
outputs the processed signal to the stages subsequent 
to the personal computer. 

The flowcharts shown in Figs. 9, 10, 11 and 
12 according to embodiments of the invention can be 
stored in any of various media (not shown) such as a 
magnetic disk, an optical disk and a semiconductor 
memory in the form of a program code readable by the 
computer or the microprocessor, and the program is read 
from the recording medium by the DSP 40 or the personal 
computer 43 thereby to execute the steps of the 
flowchart. Also, the program code can be downloaded to 
the DSP 40 or the personal computer 43 through a 
communication channel (not shown) to carry out the 
signal processing method according to the invention. 



It will thus be understood from the foregoing 
description that according to this invention, even in 
the case where the green component signal level is 
different between odd and even horizontal scanning 
lines due to the variations of the CCD characteristics 
or the performance deterioration thereof, the image 
quality deterioration can be prevented and a clear 
image can be produced without causing a horizontal 
striped pattern or speckles on a picked-up image or a 
part of image by the imaging light from the object by 
preventing the unnecessary level difference from 
occurring between horizontal lines or reducing such 
level difference. 



